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(57) Abstract 

The present invention relates to a method and apparatus for determining visually noticeable differences between two input images. A 
first contrast image is generated from the first input image and a second contrast image is generate from the second input image. A contrast 
difference image is then formed from the two contrast images. Pixels in the contrast difference image are selected which differ in intensity 
from corresponding pixels in the first contrast image by a threshold value which is referenced to the intensity of the corresponding pixels 
in the first contrast image. The selected pixels may be applied to control the performance of television signal compression circuits so as to 
control the image fidelity. 
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Determining Visually Noticeable Differences Between Two images. 

The present invention relates to determining visually noticeable differences 
between two images. The invention has particular application to the encoding 
and later decoding of an image in such fields as television signal generation, 
compression and transmission. When encoding an original image for 
television transmission and display at a receiver, it is important to know how 
well the system for transmitting and receiving will perform in reproducing the 
original image. The performance of the system is related to the visually 
noticeable differences between the original image and the displayed image. 

A number of models have already been proposed to predict whether a human 
observer will be able to discriminate between two images. One such visual 
discrimination model is known as the Sarnoff visual discrimination model. A 
visual discrimination model may take, as input, a pair of images and provide 
an output which is a map showing the probability, as a function of position on 
the images, that an observer would be able to detect differences between the 
images. The map is referred to as a JND map showing Just Noticeable 
Differences. This JND map can itself be presented as an image, with higher 
grey levels corresponding to higher probabilities of discrimination. 

In an encoding algorithm to encode picture information using quantisation 
methods, the image fidelity should be directly proportional to the quantisation 
sampling density. This assumption may be erroneous with the result that 
encoding bits are squandered where they have little impact on the fidelity of 
reproduction of an image. A better method of controlling quantisation 
parameters is to base control on a JND map to obtain either more uniform 
image fidelity for a fixed bit rate of encoding or better bit rates for a desired 
level of fidelity. 

It is an object of the present invention to improve the identification of 
noticeable differences between two images. 
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According to the present invention there is how provided a method of 
identifying visually noticeable differences between a first input image and a 
second input image, the method comprising the steps of: generating a first, 
reference contrast image from the first input image; generating a second 
contrast image from the second input image; forming a contrast difference 
image from the first and second contrast images; and selecting, from the 
contrast difference image, each pixel which has an intensity exceeding a 
threshold determined by the intensity of the corresponding pixel in the 
reference contrast image. 

Further according to the present invention, there is provided apparatus for 
identifying visually noticeable differences between a first input image and a 
second input image, the apparatus comprising: a generator to generate a first, 
reference contrast image from the first input image and a second contrast 
image from the second input image; a difference forming circuit to form a 
contrast difference image from the first and second contrast images; and a 
selector to select from the contrast difference image each pixel, which has an 
intensity exceeding a threshold, determined by the intensity of the 
corresponding pixel in the reference contrast image. 

The selected pixels which exceed the threshold value are indicative of the 
visually noticeable differences between the first input image and the second 
input image. Where the second input image is a compressed form of the first 
input image, the selected pixels will indicate where noticeable differences 
occur between them as a result of the compression. The degree of 
compression can thus be increased in those areas of each image where 
visually noticeable differences do not appear. 

The invention will now be described, by way of example, with reference to the 
accompanying drawings in which: 
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Figure 1 shows in block diagram form a video signal transmission apparatus 
embodying the present invention; 

Figure 2 shows a video signal frame transmitted by the apparatus of Figure 1 ; 

Figure 3 shows a control circuit included in the apparatus of Figure 1; 

Figure 4 shows a block of pixels that are processed by the circuit of Figure 3; 

Figure 5 shows a block diagram of steps used in processing signals in the 
circuit of Figure 3 to provide a map of just noticeable differences between two 
image frames; 

Figure 6 shows a graph of threshold values applicable to the process steps in 
Figure 5; 

Figure 7 shows a conventional television signal encoder apparatus; 

Figure 8 shows the apparatus of Figure 7 modified according to the invention 
to add a control circuit; 

Figure 8A shows details of the control circuit added into the apparatus of 
Figure 8; 

Figure 9 shows a further television signal encoder modified to incorporate the 
control circuit of Figure 8A; and 

Figure 10 shows a statistical multiplexer apparatus incorporating a plurality of 
the control circuits of Figure 8A. 

Figure 1 shows a video signal transmission apparatus including an analogue 
to digital converter 10 to receive an input analogue television signal 11 for 
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conversion to a digital signal. The input digital signal is encoded and 
compressed in an encoder 12 and applied to a transmission circuit 13. The 
transmission circuit 13 broadcasts the compressed signal in conventional 
manner for reception by one or more receiver circuits 14. The receiver circuit 
14 applies the received signal to a decoder 15 for decoding and supply to a 
display 16. 

Figure 2 shows a frame 10 included in the digital signal output from the 
encoder 12. As is well understood in the digital television art, the digital 
television signal includes a sequence of frames 10. Each frame consists of a 
matrix of pixels divided into M x N blocks of pixels. 

The processes of encoding, compression, transmission and display all 
introduce some degree of unwanted noise into the picture information 
displayed by the television display 16. At the display 16, the noise may be 
more noticeable in some picture blocks of the image than in others. A control 
circuit 30 is incorporated into the encoder 12 to control the compression of 
the television signal so as to reduce the effect of the unwanted noise. The 
control circuit 30 is shown in further detail in Figure 3. 

In Figure 3, the digital television signal from the analogue to digital converter 
10 is applied to a compression circuit 31 for compression and supply to an 
output terminal 38 connected to the transmission circuit 13. The digital 
television signal is also supplied to a contrast image generator 33. The 
generator 33 generates a contrast image from the pixels of each frame and 
applies the contrast image to a difference circuit 35. 

The compressed signal at the output terminal 38 is supplied to a 
decompression circuit 34 where the signal is subject to decompression and 
the resulting decompressed signal is passed to a 1 - frame store 32. The 
output of the 1 - frame store 32 is applied to a second contrast image 
generator 36. The difference circuit 35 receives the contrast images from the 
two contrast image generators and produces a contrast difference image. The 
contrast difference image is mapped in a JND flust noticeable difference) 
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map circuit 37. The map circuit 37 applies an output signal to control the 
compression circuit 31. 

The operation of the control circuit shown in Figure 3 will now be described 
with reference to Figures 2, 4, 5 and 6. Each frame of the digital television 
signal applied to the contrast image generator 33 and the compression circuit 
37 includes the blocks of pixels already mentioned and shown in Figure 2. In 
the contrast image generator 33, each of the pixels is processed in relation to 
a 3 x 3 neighbourhood area as shown in Figure 4. Each pixel a it j is included in 
a 3 x 3 area extending from the pixel a M to the pixel a i+ i,j+i. The image 
generator 33 generates a contrast value Cjj for each pixel a i( j according to the 
relation; 

Cj j = 1 00 * Lmax - Lmin 
Lmax + Lmin 

Where Lmax is the maximum luminance value of the pixels in the 3x3 
neighbourhood area and L min is the minimum luminance value of the pixels 
in the 3x3 neighbourhood area. 

Referring to Figure 5, a signal frame I of the digital television signal is 
received by the contrast image generator 33 in step 50 at the same time that 
a signal frame II is received by the contrast image generator 36 in step 51. 
The signal frame I is converted in step 52 into a contrast image I as already 
described above with reference to Figure 3. The signal frame II is a 
reconstruction of the television signal frame which precedes the signal frame 
I. The signal frame II is derived by the decompression circuit 34 and supplied 
to the 1 - frame store 32. The signal frame II, supplied in step 51 , is converted 
in step 53 into a contrast image II by the contrast image generator 36. The 
contrast image generator 36 uses the same process as the contrast image 
generator 33 to generate its contrast image. 

The difference circuit 35 compares the contrast value of each pixel in the 
contrast image I with the contrast value of each pixel in the contrast image II. 
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The result is a contrast difference image in which the contrast image 
differences for each frame are mapped in comparison with the preceding 
frame in the television signal. 

The contrast image I is applied, in step 54, to the JND map circuit 37. The 
contrast difference image is applied, in step 55, to the JND map circuit 37. 
The contrast image I is used as a reference for each pixel of the contrast 
difference image so as to select pixels from the contrast difference image in 
each frame as will be explained with reference to Figure 6. 

In Figure 6 there is shown a threshold curve Ct plotted against a difference 
value Ac along one axis and a reference value C along the other axis. Each 
axis has a logarithmic scale. 

The value Ac is the difference value of a pixel in the contrast difference image 
supplied by the difference circuit 35. The reference value C is the pixel value 
of the corresponding pixel in the reference contrast image supplied by the 
contrast image generator 33. It will be observed that the threshold curve Ct 
includes a constant portion for which AC approximates to 1 for values of log C 
between 0 and 3. For values of log C in excess of 3, the curve Ct has a 
gradient of 0.87. 

For each pixel where the value of log AC exceeds the threshold Ct, a control 
signal output is supplied to an output terminal 47 from the JND map circuit 37. 
The control signals are supplied to the compression circuit 31 to control the 
compression of the television signal from the analogue to digital converter 10. 
The signal compression can thereby be varied in a selective manner to 
achieve a better image fidelity for a fixed bit rate of encoding in the 
compression circuit 31 or better bit rates for a desired level of fidelity. 

Further details regarding the manner in which the compression may be 
selectively varied will now be described with reference to Figures 7 and 8. In 
Figure 7, a conventional video encoder comprises an input buffer 70 to 
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receive a digital video signal. The buffer 70 applies sliced and reordered 
picture frames to a summer 71 . The summer 71 has an output connected to a 
discrete cosine transform circuit 72 in which a discrete cosine transform is 
performed on the output from the summer 71. The transform circuit 72 is 
connected to a quantisation circuit 73, which feeds a variable length coder 74. 
The variable length coder 74 supplies an output buffer 75. The output from 
the buffer 75 is an MPEG compressed bit stream representing the digital 
video signal applied to the input buffer 70. 

A reverse quantisation circuit 76 receives an input from the quantisation 
circuit 73. The circuit 76 is connected to feed a reverse transform circuit 77, 
which performs a reverse discrete cosine transformation. Each frame of the 
input digital video signal is decompressed by the circuits 76 and 77 and 
applied to a frame store 78. A forward motion estimation circuit 79 derives 
forward motion estimation signals by comparing each frame in the store 78 
with the frame at the output from the buffer 70. 

A forward motion compensation circuit 80 receives the frame stored in the 
frame store 78 and the motion estimation signals from the circuit 79. The 
motion compensation circuit 80 applies motion compensation to the video 
signals passing through the summer 71 . 

In Figure 8, the apparatus of Figure 7 is shown adapted to include a control 
circuit 82. The control circuit 82 receives at input 84 the decompressed and 
motion compensated signal generated by the circuit 80. The circuit 82 also 
receives at input 86 the picture information from the buffer 70 in respect of the 
next succeeding frame. The circuit 82 has an output terminal 88 connected to 
the transform circuit 72. 

Figure 8A shows further detail of the control circuit 82. As seen in Figure 8A, 
the input terminal 84 is connected to a first contrast image generator 85 and 
the input terminal 86 is connected to a second contrast image generator 87. 
The two contrast image generators 85 and 87 correspond respectively to the 
two contrast image generators 33 and 36 of Figure 3. The contrast image 



SNSOOC1D: <WO 99*3161A1 I > 



WO 99/43161 8 PCT/GB99/00487 

generators 85 and 87 generate contrast images in the same manner as the 
corresponding contrast image generators 33 and 36 of Figure 3. The contrast 
images generated by the generators 85 and 87 are applied to a difference 
circuit 89, which corresponds to the difference circuit 35 in Figure 3. The 
output from the difference circuit 89 is applied to a JND map circuit 90 
corresponding to the map circuit 37 of Figure 3. 

The elements of the control circuit 82 operate in the same manner as the 
corresponding elements in the control circuit of Figure 3 to generate control 
signals at the output terminal 88. The control signals are applied to control the 
transform circuit 72 such that DCT values are sent when the control signal 
exceeds the threshold Ct but are not sent when the control signal is below the 
threshold Ct. 

In Figure 9, a second adaptation of the apparatus of Figure 7 is shown. In this 
second adaptation, the quantisation circuit 73 receives the output from the 
transform circuit 72 by way of a delay circuit 91. The control circuit 82 has its 
input terminal 84 connected to a frame store 92, its input terminal 86 
connected to the input buffer 70 and its output terminal 88 connected to the 
quantisation circuit 73 and the inverse quantisation circuit 76. A quantisation 
circuit 94 receives the output from the transform circuit 72 and supplies a 
quantised signal to an inverse quantisation circuit 96. The output from the 
inverse quantisation circuit is subject to an inverse DCT transformation in an 
inverse transform circuit 98 and is supplied to the frame store 92. 

The control circuit 82 generates a control signal on the terminal 88 to vary the 
step size of the quantisation applied by the quantisation circuit 73. The 
threshold Ct determines the quantisation parameters and hence the image 
fidelity of the MPEG bitstream supplied from the output buffer 75. 

In Figure 10, a series of encoders 1 10a to 1 10n each receives a digital video 
signal at a respective input terminal 111a to 111n. Each of the encoders 110 
quantises and encodes the respective input digital signal for supply to a 
statistical multiplexer 112. Only two encoders are shown in Figure 10 but it 
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will be appreciated by those skilled in the art that multiple encoders may be 
linked to a common statistical multiplexer 112. The bit rate of each encoder is 
a function of the input digital signal and each encoder places a varying 
demand on the bit rate capacity of the statistical multiplexer 112. The bit rate 
capacity of the multiplexer 1 12 is shared between the encoders 1 10a to 1 10n 
in a known manner. 

Control circuits 114a to 114n are provided for controlling the encoders 110a 
to 1 10n. Each of the control circuits 1 14a to 1 14n has the same configuration 
and function as the control circuit shown in Figure 3 and described above. 
The control circuits 114a to 114n each control the quantisation of the 
respective encoder to achieve a higher bit rate for a given image fidelity. The 
number of encoders 110, which share the bit rate capacity of the statistical 
multiplexer 112, may thus be increased. 

Whilst the invention has been described in relation to the control of television 
signals compressed by discrete cosine transform methods and by 
quantisation methods, it will be apparent that the invention may also be 
applied to the control of television signals compressed by other techniques 
such as wavelet compression. 

It will be seen from the above description that the invention provides a 
method and apparatus which is able to identify areas in an image where 
visually noticeable differences occur between an original image and a 
reproduction of that image. The performance of the system that provides the 
reproduction can be derived by means of the invention and adjustment made 
to the system to enable the requisite image fidelity to be achieved. Thus, 
where the system compresses the original image, the degree of compression 
can be adjusted to reduce or eliminate the usually noticeable differences. 
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CLAIMS 

1. A method of identifying visually noticeable differences between a first 
input image and a second input image, the method comprising the 
steps of: 

generating a first, reference contrast image from the first input 

image, 

generating a second contast image from the second input 

image; 

forming a contrast difference image from the first and second 
contrast images; and 

selecting, from the contrast difference image, each pixel which 
has an intensity exceeding a threshold determined by the intensity of 
the corresponding pixel in the reference contrast image. 

2. A method as claimed in claim 1, wherein the first and second input 
images each consist of a matrix of pixels within a television signal 
frame. 

3. A method as claimed claim 2, wherein the first and second contrast 
images are generated by processing each of the pixels in the 
respective frame to produce a contrast value c i(j for each pixel a tii 
according to the relation; 

Cjj = 100 * Lmax - Lmin 
Lmax + Lmin 

Where Lmax is the maximum luminance value of the pixels in the 3 x 3 
neighbourhood area adjoining the pixel and Lmin is the minimum 
luminance value of the pixels in the 3x3 neighbourhood area. 

4. A method as claimed in claim 1, 2 or 3, including the further step of 
deriving the second input image from the first input image by first 
compressing and subsequently decompressing the first input image. 
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5. A method as claimed in claim 4, wherein the step of deriving the 
second input image from the first input image comprises compressing 
the first input image by a discrete cosine transformation step and 
decompressing the compressed first input image by an inverse discrete 
cosine transformation step. 

6 A method as claimed in claim 4, wherein the step of deriving the 
second input image from the first input image comprises compressing 
the first input image by a digital quantisation step and decompressing 
the compressed first input image by an inverse digital quantisation 
step. 

7. A method as claimed in claim 5 or 6, comprising the further step of 
controlling the degree of compression by reference to the pixels 
selected from the contrast difference image, so as to limit the number 
of visually noticeable differences between the first and second input 
images. 

8. Apparatus for identifying visually noticeable differences between a first 
input image and a second input image, the apparatus comprising: 

a generator to generate a first, reference contrast image from 
the first input image and a second contrast image from the second 
input image; 

a difference forming circuit to form a contrast difference image 
from the first and second contrast images; and 

a selector to select from the contrast difference image each 
pixel, which has an intensity exceeding a threshold, determined by the 
intensity of the corresponding pixel in the reference contrast image. 

9. Apparatus as claimed in claim 8, wherein the generator is adapted to 
generate the first and second contrast images from first and second 
input images each consisting of a matrix of pixels within a television 
signal frame. 
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10. Apparatus as claimed in claim 9, wherein the generator is adapted to 
generate the first and second contrast images by processing each of 
the pixels in the respective frame to produce a contrast value Cjj for 
each pixel a^ according to the relation; 

Cjj = 1 00 * Lmax - Lmin 
Lmax + Lmin 

Where Lmax is the maximum luminance value of the pixels in the 3 x 3 
neighbourhood area adjoining the pixel and Lmin is the minimum 
luminance value of the pixels in the 3x3 neighbourhood area. 

1 1 . Apparatus as claimed in claim 8, 9 or 10, further including compression 
and decompression circuits adapted to derive the second input image 
from the first input image by first compressing and subsequently 
decompressing the first input image. 

12. Apparatus as claimed in claim 11, wherein the compression circuit 
comprises means to compress the first input image by discrete cosine 
transformation and the decompression circuit comprises means to 
decompress the compressed first input image by inverse discrete 
cosine transformation. 

13. Apparatus as claimed in claim 11, wherein the compression circuit 
comprises means to compress the first input image by digital 
quantisation and the decompression circuit comprises means to 
decompress the compressed first input image by inverse digital 
quantisation. 

14. Apparatus as claimed in claim 11 or 12, further comprising control 
means to control the compression in the compression circuit in 
response to the pixels selected from the contrast difference image, so 
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as to limit the number of visually noticeable differences between the 
first and second input images. 

15. Signal encoding and multiplexing apparatus comprising a plurality of 
signal encoders for encoding and compressing a repetitive plurality of 
input digital television signals to produce a corresponding plurality of 
encoded, compressed television signals and a statistical multiplexer to 
multiplex the encoded television signals, wherein each of the signal 
encoders includes a control circuit to control the degree of 
compression applied by the signal encoder, the control circuit being as 
claimed in claim 14. 

16. A method of identifying visually noticeable differences between a first 
input image, the method being substantially as herein before described 
with reference to Figure 1, 3, 8, 9, or 10 of the accompanying 
drawings. 

17. Apparatus for identifying visually noticeable differences between a first 
input image and a second input image, the apparatus being 
substantially as herein before described with reference to Figure 1, 3, 
8, 9, or 10 of the accompanying drawings. 
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